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1.  Introduction 

There  are  a  variety  of  point  processes  associated  with  the  jump 
times  of  a  Markov  process.  For  instance,  in  a  Markovian  network  of 
queues,  one  might  be  interested  in  the  point  process  of  times  at  which 
units  move  between  two  sectors  of  the  network.  More  generally,  if  the 
network  has  synchronous  movements  of  items,  one  might  be  interested  in 
the  marked  point  process  of  the  times  at  which  batches  of  units  move 
between  two  sectors  and  the  numbers  of  units  in  the  batches  (the  batch 
size  being  the  "murk"  of  the  time  of  the  movement).  One  can  formulate 
such  a  point  process  as  a  functional  of  the  Markov  process  representing 
the  network.  The  typical  aim  is  to  describe  the  behavior  of  the  point 
process  in  terms  of  the  characteristic  of  the  Markov  process.  Some 
immediate  questions  in  this  regard  are:  Is  such  a  point  process  Poisson 
(or  marked  Poisson)?  Is  a  collection  of  these  point  processes 
multi-variate  Poisson  (or  marked  Poisson);  and  wliat  are  the  dependencies, 
if  any,  among  them? 

These  are  the  issues  this  study  addresses.  We  begin  in  Section  2  by 
presenting  conditions  under  which  a  point  process  of  certain  jump  times 
of  a  Markov  process  is  a  Poisson  process.  It  is  well  known  that  a  simple 
point  process  on  the  real  line  is  Poisson  with  rale  a  if  its  compensator 
lias  the  constant  intensity  a  (Theorem  2.3(i)).  We  present  a  reverse-time 
version  of  this  (Theorem  2.3(ii)).  It  says  that  if  the  Markov  process  is 
stationary  and  the  compensator  of  the  point  process  in  reverse  lime  lias 
the  constant  intensity  a.  then  the  point  process  is  Poisson  with  rate  a. 
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This  is  an  easy-to-use  criterion  for  establishing  whether  a  point  process 
of  jump  times  is  Poisson.  We  also  give  necessary  conditions  for  this 
Poissonness.  In  Section  3,  we  present  similar  conditions  under  which 
more  general  functionals  of  a  pure  jump  Markov  process  are  marked  Poisson 
processes.  Some  of  our  results  overlap  those  of  Melamed  (1979),  Brerruud 
(19S1),  Variaya  and  Walrand  (1981).  and  Disney  and  Kiessler  (1987). 
Melamed  and  Disney  and  Kiessler  derive  their  results  using  a  Markov 
renewal  argument  and  Brcmaud,  Variaya  and  Walrand  use  a  filtering 
argument.  We  use  a  simpler  approach  based  on  the  notion  of  the  time 
reversibility  of  the  compensator  of  a  point  process.  This  approach  lays 
bare  the  characteristics  underlying  the  Poisson  property,  and  it  readily 
extends  to  the  more  general  settings  in  Sections  3-f>. 

The  applications  in  this  area  have  been  primarily  for  queueing 
systems.  Burke  (19:36)  and  Reich  (1997)  showed  that,  in  a  stationary 
M/M/1  queueing  system,  I  he  output  flow  is  a  Poisson  process  with  the  same 
rate  as  the  Poisson  input  flow.  Similarly,  the  exit  flows  from  the 
queues  in  a  Jackson  network  are  independen t  Poisson  processes;  this  is 
discussed  in  the  references  in  the  preceding  paragraphs  and  in  Kelly 
(1979).  Disney  and  Konig  ( 198S)  and  Whittle  (I9S6).  In  Sections  2—4,  we 
discuss  applications  identifying  Poisson,  compound  Poisson  and 
mu  1 1 i var ia te-Poi sson  flows  in  single  queues  and  in  queueing  networks  with 
dependent  nodes.  We  end  in  Section  f»  by  indicating  how  our  results 
extend  to  Markov  processes  with  general  state  spaces  and  to  functionals 
of  semi-Markov ian  processes. 


I# 

i 


2.  Jump  Times  of  a  Markov  Process  that  Form  a  Poisson  Process 
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Let  X  =  {X  •'  (  C  R)  be  a  Markov  process  with  a  countable  state  space 


9  and  transition  rates 


q(x.y)  =  lim  P{X  =  y  |  Xn  =  x)/t .  x  ^  y. 

t  lo  1  ° 


and  q(x.x)  =  0.  We  indicate  later  how  our  results  extend  to  a  general 


state  space.  We  adopt  the  standard  assumption  that 


q ( x )  •'  =  2  q(x.y)  <  °» .  x  C  .}, 

y 


and  that  each  sample  path  of  X  is  right  continuous  and  has  a  finite 


number  of  jumps  in  any  finite  time  period.  Then  the  sojourn  time  of  X  in 


a  state  x  is  exponential  with  mean  q(x)  and.  at  the  end  of  the  sojourn. 


X  jumps  to  some  state  y  with  probability  q(x.y)/q(x).  y€9.  For 


convenience,  we  assume  t  lia  t  X  is  irreducible. 


We  shall  study  the  point  process  N  on  R  defined  by 


(2.1) 


N(A)  =  X  f(X  X  ) 
tCA 


where  A  is  a  Hotel  set  in  R  and  f:  .9  x  9  {0,1}  with  f(x.x)  =0,  x  C  :t . 


The  N(A)  is  the  number  of  jumps  of  X  from  some  x  to  some  y  for  which 


f(x.y)  =  1  in  the  time  period  A.  Any  such  f  is  ;m  indicator  function 


f(x.y)  =  1  ( ( x .  y )  C  S)  of  a  subset  S  of  9  x  9  ilint  does  not  contain  pairs 


of  identical  values;  the  N(A)  would  then  be  the  number  of  transitions  of 


X  in  the  period  A  that  take  place  in  the  transition  set  S.  Clearly  N(A) 


is  finite  when  A  is  bounded  and  it  may  be  infinite  when  A  is  unbounded . 


We  shall  frequently  use  the  function 


(2.2) 


a(x)  =  >:  q ( x  , y  ) f  ( x  . y  ) .  x  C  9. 

y 


First  note  tlvui  the  me  art  measure  of  N  is 
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(2.3) 


EN(A)  =  /.Ea(X  )dt 

A  t 


which  is  Levy's  formula  (see  for  instance  Benveniste  and  Jacod  (1973)). 
When  X  is  stationary,  an  easy  check  shows  that  N  is  stationary  (i.e.  for 

any  A^ . A^.  the  distribution  of  N(A^+t) .  N(A^+t)  is  independent  of 

t).  Consequently.  EN(s,t]  =  a(t-s).  s  <  t  in  R,  where 


(2.4) 


a  =  EN  ( 0 ,  1  ]  =  2  7r(x)q(x.y)f(x,y) 

x.y 


and  jt(x)  =  P(X^=x}  is  the  equilibrium  (or  stationary)  distribution  of  X. 
Here  0  <  a  <  Also,  with  probability  one,  t  ^N(s.s+t]  -*  a  as  t  -* 

One  can  use  N  to  model  a  variety  of  event  occurrences  of  X  by 
appropriate  selections  of  f.  For  instance,  suppose  X  takes  values  in 
1  =  Z*.  the  m-dimensional  vectors  with  nonnegative  integer-valued 
entries.  Then  N  records  the  downward  jumps  of  X  when  f(x,y)  =  1  iff 

x  ?  y  and  x^  >  y^.  j  =  l . m.  Similarly,  N  records  the  jumps  of  X  at 

which  the  maximum  component  does  not  change  when  f(x,y)  =  1  iff  x  t  y  and 
max  x .  =  max  y . . 

J  J  J  J 

We  shall  investigate  conditions  under  which  N  is  a  Poisson  process. 
We  say  tltat  the  future  of  N  is  independent  of  the  past  of  X.  denoted 
N+  -U-  X_  ,  if  (N(A):  A  C  [t,®)}  is  independent  of  (X^:  u  <  l }  ,  t  C  R. 

({X^:  u  <  l)  cat)  be  replaced  simply  by  X^  since  X  is  Markovian). 
Similarly.  N_  X+  denotes  that  the  past  of  N  is  independent  of  the 
future  of  X.  Our  first  result  is  that  these  conditions  are  sufficient 
for  N  to  be  a  Poisson  process;  the  N  might  be  a  non-stai ionary  Poisson 
process.  We  write  "N  is  !’/*( a)"  to  me;ut  llial  N  is  a  stationary 


I 
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(time-homogeneous)  Poisson  process  with  rate  a.  The  degenerate  case  a  = 

0  corresponds  to  N  =  0;  the  case  a  =  °>  is  not  possible. 

Theorem  2.1.  If  N+  X_  or  N_  X+.  then  N  is  a  Poisson  process.  In 
this  case.  N  is  f(&)  if  and  only  if  EafX^)  =  a.  t  €  R. 

Proof .  The  N  is  a  Poisson  process  if  it  is  simple  (i.e..  N{ { t } )  =  0  or 
1.  t  €  R),  it  has  no  fixed  atoms  and  has  independent  increments  (i.e., 

N(Aj) . N(An)  are  independent  for  disjoint  Aj . A^);  see  p.  58  of 

Kallenberg  (19S3).  Since  the  probability  is  zero  that  X  has  a  jump  at 
any  specified  time,  it  follows  tliat  N  is  simple  with  no  fixed  atoms.  Now 
suppose  N+  -ii  X__.  Then,  for  any  s  <  t  in  R, 

P{N(s,  t]  =  n  |  N(A)  :  A  C  (-«>.s]} 

=  E[P{ N( s. t]  =  n  j  Xp  :  r  <  s}]  =  P{N(s,t]  =  n). 

Thus  N  has  independent  increments  and  hence  N  is  Poisson.  This 
conclusion  also  follows  when  N_  X+  since 

P{ N( s ,  t  ]  =  n  |  N(A)  :  A  C  [t  ,“>)} 

=  E[P{N(s.l]  =  n  |  Xu  :  u  >  t }]  =  P{N(s,t]  =  n). 

The  second  assertion  on  the  slaiionarity  of  N  follows  from  (2.3).0 
Remark  2.2.  From  the  proof,  it  is  clear  that  the  first  assertion  of 
Theorem  2.1  is  true  for  any  pure  jump  stochastic  process  X;  the  Markovian 
property  is  used  only  for  the  second  assertion.  Also,  both  assertions 
are  true  when  X  is  a  Markov  process  that  is  not  time  homogeneous  (its 
transition  rates  are  lime  dependent).  Melamed  (1979)  showed  tltat  N+  -U-  X_ 
implies  that  N  is  Poisson;  his  argument  relics  on  the  property  dial  X  ;md 
{(N(s,tJ.  X^):  l  >  s}  are  Markovian. 


We  now  discuss  conditions  on  the  parameters  of  the  Markov  process  X 


under  which  N  is  :f(a).  When  X  is  stationary  with  equilibrium 


distribution  v .  we  shall  frequently  use  the  function 


(2.5) 


«*(*)  =  "(*)  1  2  7T  ( y  )q(y  ,x)f  (y  ,x) .  x  C  it. 


Theorem  2.3.  (i)  If  a(x)  =  a.  x  C  :t.  then  N+  X_  and  N  is  ^(a) .  ( i  i  ) 

44 

If  X  is  stationary  with  equilibrium  distribution  it .  and  «  (x)  =  a.  x  C  't . 
then  N_  X+  and  N  is  if  (a). 

Proof .  Assertion  (i)  is  a  special  case  of  Theorem  18.9  in  l.iptser  and 
Shiryayev  (1978)  (or  Theorem  T8  in  Bremntid  (1981)),  which  says  that  a 
simple  point  process  on  R  is  "/‘(a)  if  its  nmpcnsntor  has  the  non-r;uidom 
intensity  a.  In  our  sett  inn.  for  each  s  C  R.  the  process 
(2.9)  Mt  =  N(s.s  +  t]  -  /*+'  o(Xu)du 

is  art  -mar  t  ingale,  where  =  o(^u:  s  <  u  <  s  +  t);  the  process 

A(  =  1  a(X^)du  is  the  compensator  of  N(s,  s  +  t]  ;ind  a(X( )  is  the 

intensity  of  this  compensator. 

44  _ 

To  prove  assertion  (ii).  consider  the  process  X^  =  X  .  t  C  R.  where 

—  44 

X{  =  .  The  X  is  the  right -coni inuous  t  ime-reversa  1  of  X.  Luc  It 

w 

sample  path  of  X  is  the  stunc  as  :>  sample  |xith  of  X  traversed  in  the 
opposite  direction,  and  vice  versa.  Since  X  is  stationary,  it  follows 


that  X  is  ait  irreducible,  stationary  Markov  process  and  its  transition 


rates  are 


44  _  | 

q  (x.y)  =  ir(x)  tr(y)q(y.x).  x.y  C  :t  . 

44 

iJefine  the  point  process  N  on  R  by 

44  mu  v 

N  (A)  =  >:  r  (X  .  x  ) 

t  C  A 

where  f  (x.y)  =  f(y.x).  x.y  C  S .  dearly  N  (A)  =  N(-A),  for  each  A.  and 


& 

a 


so  N  is  tiie  time-reversal  of  N.  Consequently,  N  is  't( a)  if  and  only  if 
N*  is  #(a) . 

4*  M  M  H 

Now  observe  that  a  (x)  =  5  q  (x.y)f  (x.y).  Tliat  is,  a  is  the 

y 

^  ^  MM 

function  a  for  the  processes  X  .  N  .  Then  an  application  of  (i)  to  X  ,N 
says  that  N+  X_  and  N  is  ?l'( a).  But  these  statements  are  eqi valent  to 
the  respective  statements  that  N__  X+  and  N  is  'f(a).  since  X  ,  N  are 
the  time  reversals  of  X.N.O 

Remarks.  (1)  In  Theorem  2.3.  statement  (i)  is  the  well-known 
property  of  Markov  processes  tliat  N  is  t’f'(a)  if  the  intensity  u(X()  of  N's 
compensator  equals  the  constant  a.  Statement  (ii)  is  simply  a  reversed 
time  version  of  (i):  it  is  (i)  in  terms  of  X  and  N  viewed  in  reverse 

M  MM 

time.  The  a  is  the  reversed  time  version  of  a  in  the  sense  tliat  a  (X^) 
is  the  intensity  of  the  componsn tor  of  N  in  reverse  time.  Also.  N_ X+ 
in  (ii)  is  the  time  reversal  of  N  ^  X_  in  (i).  (2)  Note  that  the 

process  X  in  (i)  may  be  transient,  recurrent  or  non-slat  ionary ;  but  in 
(ii).  X  must  be  stationary.  (3)  In  consul  line;  the  references  for 
Theorem  2.3(i).  one  can  see  that  this  result  is  true  for  a  non-Markov ian 
process  X  for  which  a-  $  -*  K+  is  a  function  such  tliat  the  process  in 
(2.6)  is  an  >  ^  -mar  l  i  ni^a  le .  Similarly,  it  follows  that  Theorem  2.3(ii)  is 
true  for  a  non -Markovian  process  X  if  a  ■  :i  ->  is  a  function  such  that, 
for  each  s( R . 

M*  =  N[s- 1  ,  s  )  -  u*(X(|)du 

M 

is  an  £  -mar  l  i  titta  1  e  where  ft  -  o(X  :  s-(  <  u  <  s). 

t  l  '  u  _  ’ 


vy  «• 


» 


In  applications,  the  Poisson  property  of  N  due  to  u(x)  =  a.  x  C  :J . 
is  usually  foreseen,  while  the  Poisson  properly  of  N  due  to  a  (x)  =  a,  x 
C  ;t.  might  not  be  anticipated  Here  is  an  example. 

Example  2.4.  M/M/1  and  Hatch  Service  Queues.  Sup|x>se  the  Markov  process 

X  lias  the  state  space  /.+  and  transit  ion  rates 

q(n.n+l)=A  n  C  / 

q(  n,  n  -  K)  =  fj  n  >  k 

q(n.O)  =  p  n  <  K 

where  A,  p  and  k  are  positive.  'Hus  process  represents  the  number  of 
customers  in  a  queueing  system  in  which  customers  arrive  singly  at  the 
rate  A  and  are  served  in  batches  such  that  when  n  >  k  customers  are  in 
the  system,  then  batches  of  k  customers  tkqvirt  at  the  rate  p:  and  when 
n  <  k  customers  are  present,  then  all  of  the  customers  depart  at  the  rate 
p.  When  k  =  1.  this  is  the  M/M/ I  queueing  system. 

Implicit  in  the  description  of  this  queueing  system,  the  point 
process  N  of  customer  arrivals  is  ‘■'I'(X)  .  regardless  of  whether  X  is 
transient  or  recurrent.  Indeed,  this  follows  from  Theorem  2.T  (i)  since 
N  is  defined  by  (2.1)  with  i'(n.n')  =  1  iff  n'  =  n  +  1  and 

«(n)  -  y.  q  ( n  ,  n  '  )  f  ( n,  n  ’  )  -  q(n,n  +  1)  =  A. 
n ' 

Now.  suppose  that  N  denotes  the  point  process  of  limes  at  which 
batches  of  size  k  depart  from  the  system.  This  is  defined  by  (2.1)  with 
f(n.n')  -  1  iff  n'  =  n  -  k  and  n  /  k.  In  this  case. 

o(n)  =  q(n.n  -  k)l(n  >  k)  =  pl(n  >  k) . 

This  depends  on  n.  and  so  Theorem  2..T  (i)  does  not  ensure  that  N  is 
Poisson.  However,  assume  that  X  is  stationary.  Necessarily.  A  <  |ik  and 
the  equilibrium  distribution  of  X  is 


7r(n}  =  r(l-r),  n  >  0 . 
whore  r  €  (0,1)  is  the  unique  root  of 

pr  -  (X  +  m)t  +  X  =  0 

(see  Section  3.2  of  Cross  and  Harris  (19ST>)).  Clearly 

a  (n)  =  7r(n)  *  I  ir( n '  )q(n  ' .  n)  f  ( n  '  ,  n ) 
n ' 

=  ir( n)  '  7r(n  +  K)q(n  +  K.n) 

=  pr^  =  p  +  X(  1  -  1/r ) . 

Thus .  we  conclude  by  Theorem  2.3  (ii)  that  the  process  N  is 

3P(p  +  X(  1  -  1/r)).  One  would  probably  not  anticipate  this  result  from 

the  description  of  the  process,  or  even  from  earlier  work  in  this  area. 

For  the  special  case  in  which  X  is  the  M/M/1  queue,  we  have  K  =  1, 

r  =  A/p,  and  so  N.  which  is  the  departure  process,  is  :f(X).  Burke  (  1  056  ) 

and  Reich  (1957)  were  the  first  ones  to  prove  this. 

Example  2.5.  Queues  With  Compound  Poisson  Arrivals  and  Poisson 
Departures.  Suppose  the  Markov  process  has  the  state  space  Z+  and 


transi t ion  rates 


q(n,  n  +  m)  =  X^  p  ( 1  -  p) 

q(n .  n  -  1)  =  U 


m  >  1 ,  n  C  7. 


n  >  1 


where  X  ,  p  are  positive  and  0  <  p  <  1.  This  process  represents  the 
n  n 

number  of  customers  in  a  queueing  system  in  which  batches  of  customers 
arrive  at  the  rale  X^  when  n  customers  are  present  and  the  number  of 
customers  in  a  batch  has  a  geometric  distribution  with  parameter  p.  The 
customers  depart  at  the  rate  /i  when  n  ate  in  the  system.  The 
equilibrium  distribution  of  X  is 

-1  -1  n_1 

ir(n)  =  w(0)X(yjj  .  .  ,p()  17  (Xj.  +  ppk).  n  >  1  , 


provided  the  sum  of  these  terms  over  n  is  finite,  which  we  assume  is  true 
(see  Kook  (1988)). 

Suppose  that  N  is  the  point  process  of  customer  departures.  This  is 
defined  by  (2.1)  with  f(n.n')  =  1  iff  n‘  =  n  -  1.  Clearly, 

a*(0)  =  Tr(O)-1  rr(  1  )q(  1.0)  -  XQ 

*  _  J 

a  (n)  =  ir( n)  iT(n  +  l)q(n  +  l.n)  =  X^  +  pp^,  n  >  1. 

Thus ,  if  X  is  stationary  and  X,,  =  a  and  X  +  pu  =  a.  n  >  1,  then  the 

0  n  n 

departure  process  N  is  ^(a). 

We  end  this  section  with  elaborations  on  Theorem  2.3  that  establish 
necessary  conditions  for  N  to  be  Poisson. 

Theorem  2.6.  When  X  is  recurrent,  the  following  statements  are 
equivalent . 

( i  )  N+  -11-  X_  and  N  i  s  d'(a)  . 

( ii)  a(x)  =  a.  x  €  $ . 

(iii)  N+  -11-  X_  and  EN(s.t]  =  a(t-s),  s  <  t  in  R. 

(iv)  E[o(X  )  |  X  =  x]  =  a,  x€;l,  s<tinR. 

Proof .  Theorem  2.3  ensures  that  (ii)  implies  (i)-  Clearly  (i)  implies 
(iii).  Now,  if  (iii)  holds,  then,  for  s  <  t  in  R. 

a(t-s)  =  EN(s.t]  =  E[N(s, t )  |  X  =  x] 

=  E[a(X(j)  |  Xs=x]du 

the  lust  equality  being  Levy's  formula.  Taking  the  derivative  of  this 
with  resj)ect  to  t  yields  (iv).  Einally,  if  equation  (iv)  holds,  then 
taking  the  derivative  of  it  with  respect  to  l  and  letting  l  1  s,  we 
obta  in 

2  «(y)q(x.y)/q(x)  =  «(x)  for  each  x. 


This  says  that  a  is  a  harmonic  (unction  of  the  Markov  matrix 

{q(x ,y )/q{x) ) .  Now.  this  matrix  is  recurrent  under  the  assumption  that  X 

is  recurrent.  But  we  know  (see  for  instance  Section  7.2  of  Cinlar 

(197S))  that  harmonic  functions  of  irreducible,  recurrent  Markov  chains 

are  constant.  This  observation  and  equation  (iv)  imply  (ii).D 

Theorem  2.7.  When  X  is  stationary  with  equilibrium  distribution  ir .  the 

following  statements  are  equivalent. 

(i)  N_  X+  and  N  is  lf(a). 

( i  i )  a*(x)  =  a,  x  €  *1. 

(iii)  N_  X+  and  EN(0.  1]  =  a. 

(iv)  E[u*(X^)  |  X  =  x]  =  a,  x  €  A.  s  <  t  in  R. 

94 

(v)  a  (x)  =  S  a(y)7r(y)  =  a.  x  €  1. 

y 

Proof .  The  equivalence  of  (i)  -  (iv)  follows  from  Theorem  2.6  applied  to 

94  94 

the  revet sed-t ime  processes  N  ,  X  defined  in  the  proof  of  Theorem  2.3. 
Furthermore,  (v)  obviously  implies  ( i i ) .  And  (ii)  implies  (v)  since  N  is 
/'(a)  by  (i)  and  by  (2.-1),  we  have 

a  =  EN ( 0 . 1  ]  =  fl0  Ea(Xt)dt  -  2  a(y)tr(y). 

y 

Thus  (i)  -  (v)  are  equivalent  statements. □ 

94 

Remark  2.8.  Statement  (v).  relating  a  to  «,  is  the  "independence 
condition"  that  is  the  focus  of  Melamed  (1979).  Bremaod  (19S1)  and 
Varaiya  and  Walrand  (1981).  Their  main  result,  which  is  implicit  in 
Theorem  2.7,  is  as  follows:  If  X  is  stationary  with  equilibrium 
distribution  ir .  then  N_  ^  X+  if  and  only  if  a  (x)  =  X  rt(y)ir(y),  x  C  :1  . 

y 

In  this  case,  N  is  ^(a)  where  a  is  the  preceding  sum. 


'•V 


«vcw 


A 


lL 


lL 


Although  we  are  considering  processes  N  and  X  defined  on  the  entire 
real  line,  our  results  herein  also  apply  to  such  processes  defined  on  any 
time  interval  I  in  R.  Indeed,  consider  the  process  extended  to  all  of  R 
and  then  the  results  apply  to  these  extended  processes  and  lienee  to  their 
restrictions  to  I. 


3.  General  Poisson  Functionals  of  Markov  Processes 

We  have  been  studying  the  point  process  N  that  records  the  times  at 
which  the  Markov  process  X  jumps  from  some  x  to  some  y  with  f(x,y)  =  1. 
We  shall  now  study  a  more  general  point  process  that  records  additional 
information  at  these  jumps.  Suppose  lliat  h  is  a  function  from  1  x  1  to 
some  space  if.'.  Consider  the  marked  point  process  M  on  R  x  it'  defined  by 


(3.1) 


M(AxB)  =  2  f (X  _.X  )  1 ( h ( X  .X  )  C  B) 

tCA  1  * 


where  f  is  as  above.  A  x  B  is  in  the  o-fie)d  of  R  x  ;t '  and  1  ( -  G  B )  is 
the  indicator  function  of  B.  This  M  is  a  functional  of  X  that  records 
the  "mark''  h(X  .X  )  at  each  jump  lime  t  of  X  at  which  f ( X ^  ,X{)  =  1.  As 
a  point  process.  M  is  simple  and  M({t}  x  it')  =  0  or  1.  t  f  R.  Also. 

M(A  x  it')  =  N(A)  records  the  limes  at  which  M  records  the  marks. 

In  this  section,  we  identify  conditions  on  X  under  which  M  is 
Poisson  with  EM((s.t]  x  B)  =  a(t-s)F(B),  where  a  >  0  and  F  is  a 
probability  measure  on  it '  .  In  this  case,  the  process  N  of  occurrence 
times  of  the  marks  of  M,  is  i?(a).  and  each  mark  lias  the  distribution  F. 

We  call  such  an  M  a  marked  Poisson  process  with  rate  a  and  mark 
distribution  F.  and  we  simply  say  M  is  MT(a. F) . 
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The  approach  we  used  in  Section  2  for  deriving  Poisson  characteri¬ 
zations  of  N  readily  extends  to  yield  marked  Poisson  characterizations  of 
M.  The  following  results  are  analogues  of  Theorems  2.1.  2.3  and  2.7;  the 
analogue  of  Theorem  2.6  can  be  seen  from  Theorem  3.3  and  hence  is  not 
displayed.  Their  proofs  are  omitted  since  they  follow  the  same  line  of 
reasoning  as  their  counterparts  in  Section  2.  Here  we  let 
a(x.U)  =  I  q(x.y)f(x.y)i(h(x,y)  C  B) 

y 

«*(x.B)  =  tr(x)  1  2  ir(y)q(y  ,x)f(y.x)l  (h(y.x)  €  B) 

y 

Theorem  3.1.  If  M+  X_  or  M_  X+.  then  M  is  :i  Poisson  process.  In 
this  case,  M  is  j(W(a.K)  if  and  only  if  t«(X(  ,B)  =  aK(B).  for  each  t  and 


Theorem  3.2.  (i)  If  u(x.D)  =  uF(B),  for  each  x  and  B.  then  M+  X_  and 

M  is  I- } .  (ii)  If  <»  (x.B)  =  nl-(B).  for  each  x  and  B,  then  M_  -N-  X+ 

and  M  is  JH(a,F). 

Theorem  3.3.  When  X  is  stationary  with  equilibrium  distribution  n ,  then 
the  following  statements  are  equivalent. 

(i )  M_  X+  and  M  is  Jtt{ a.F). 

(ii)  a  (x.B)  =  aF(B).  for  each  x  and  B. 

(iii)  M_  &  x+  and  LM((0.1]  x  B)  =  aF{ B)  for  each  x  ;uid  B. 

(iv)  F[o  (X^.B)  |  X^=x]  =  aF(B).  for  each  x.  B  and  s  <  t  in  R. 

(v)  a  (x.B)  =  2  ur(y.B)tr(y)  =  al  (B),  for  each  x  ami  B. 

y 

Point  processes  of  the  form  (3.1)  are  useful  lor  representing 
mu  1 1 ivar iate  and  compound  point  processes.  Indeed,  suppose  one  is 

interested  in  the  n-d i mens iona 1  compound  point  process  (Mj . M  ) 

de  f  i  tied  by 


I*/ 

ly 

w 


g 


ft 

i 


I 

$ 

•V 


(3.2) 


M  .(A)  =  1  f(X,  .X  h.(X  ,X  ) 

J  l€A  l~  1  J  l~  1 


where  :  1  x  1  -»  R  (or  any  other  group).  These  processes  contain  the 
same  information  as  M  defined  in  (3.1)  with  ‘1 '  =  Rn  and 

h(x.y)  =  (hj(x,y) . h^fx.y)).  More  precisely,  there  is  a  one-to-one 

correspondence  between  point  processes  M  on  R  x  Rn  as  in  (3.1)  and 
n-dimensional  compound  point  processes  (Mj , . . . ,M  )  as  in  (3.2).  Note 

that  M  on  R  x  Rn  is  Jtf'(n.F)  if  and  only  if  its  corresponding  (M^ . M^) 

is  an  n-dimensional  compound  Poisson  process  with  rate  a  and  atom 
distribution  F  on  Rn.  In  this  case. 


P{M. (s . t ]  C  B. . M  (s. t]  €  B  } 

1  1  J  1  n'  J  n' 


2  F^B  x...xB  )ak(.-s)k  e 

A  *  ** 


k,  ,k  -a(  t-s)  , 


And  each  M  .  is  a  compound  Poisson  process  with  rale  a.  =  a(l-F.(0))  and 
J  J  J 

atom  distribution  F.,  where  F, . F  are  the  marginal  distributions  of 

J  1  n 

F.  Here  are  some  special  cases: 

Cl:  Mj . are  independent  compound  Poisson  processes  with  rates 

a. . a  and  atom  distributions  F . F  if  and  only  if  F  =  F.x...xF 

In  I  n  In 

C2:  (Mj . M^)  is  an  n-dimensional  Poisson  process  with  rate  a  and 

point  allocation  distribution  F  on  (0,l}n  if  and  only  if  F  has  support  on 

(0,l}n.  In  this  case,  each  M.  is  tf(u.). 

J  J 

C3:  M, . M  are  independent  Poisson  processes  with  rates  a . a  if 

In  r  In 

and  only  if  F  has  support  on  (Cj,...en},  where  c  is  the  jth  unit  vector 
with  1  in  entry  j  and  0’s  elsewhere. 

Kcnurk  B.d.  Fach  of  the  preceding  theorems  hold  for  these  cases  Cl.  ('2, 
C3  when  the  F  in  the  theorem  is  as  specified  in  the  case  of  interest. 

For  instance.  Theorem  3.2(H)  for  case  C3  reads:  If  a  (x,(e.))  =  a.. 


cvvvvv*. 
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j=l . m.  and  a  (x. {z})  =  0  otherwise,  x  €  3,  then  (Mj . M  X+  and 

Mj . are  independent  Poisson  processes  with  respective  rates 

a. .... .a  . 

1  n 

An  obvious  appl icat ion  of  Theorem  3.2(i)  shows  that  the  process  of 
customer  arrivals  in  Example  2.5,  when  A^  =  A,  is  a  compound  Poisson 
process  with  rate  A  and  geometric  atom  distribution  pm  ^ ( 1  — p ) .  The 
following  is  an  example  of  a  not  so  obvious  compound  Poisson  flow  in  a 
queueing  process. 

Example  3.5.  A  Batch-Service  Qjeucing  System  With  Poisson  Arrivals  and 
Compound  Poisson  Departures.  Suppose  t lie  Markov  process  has  the  stale 
space  Z  and  transition  rates 

q{ 0, 1 )  =  A(1  -  p).  q(n,n  +  1)  =  A,  n  >  1. 

q( n. n  -  m)  =  ppm  1 ( 1  —  p )  1  <  m  <  n,  n  >  1, 

q{ n , 0)  =  jjpn  *  n  >  1  , 

where  A.p  are  positive  and  0  <  p  <  1.  This  process  represents  the  number 
of  customers  in  a  queueing  system  in  which  customers  arrive  at  the  rate  A 
and  are  served  in  batches  as  follows.  When  there  are  customers  in  the 
system,  "buses"  arrive  at  a  rate  p  to  lake  them  immediately  from  the 
queue.  Busing  is  a  common  practice  in  computer  systems  and  material 
handling  systems.  The  number  of  customers  each  bus  can  take  is  a  random 
variable  with  the  geometric  distribution  p^^l-p),  m  >  1.  Also,  when 
there  are  no  customers  in  the  queue  and  a  customer  arrives,  then  with 
probability  p  there  is  a  bus  available  to  take  the  customer  without 


delay.  The  equilibrium  distribution  of  X  is 

tr(n)  =  ir(0)(  l-p)An/(;j  +  pA)1'.  n  >  1, 
provided  A  <  p  +  pA.  which  wc  assume  is  true  (see  Kook  (HISS)). 
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Consider  the  compound  point  process 

D(A)  =  2  max{0.  Xt  -  X  J 

tCA  1 

that  describes  the  total  number  of  departures  in  the  time  period  A.  It 
records  the  times  at  which  batches  of  customers  depart  and  the  batch 
sizes  as  well.  This  process  corresponds  to  M  in  (3.1)  with  f(n.n')  =  1 


iff  n‘  f t  n  and 


h(n.n‘ )  = 


n  -  n  '  n  ‘  <  n 


n'  >  n  . 


Clearly,  for  each  n  >  0  and  m  >  1, 

a  (n.{m))  =  7t(n)  ^  2  Tr(n' )q(n' ,n)f  (n' ,n)  l(h(n' ,n)  =  m) 
n 1 

=  tr(n)  V(n  +  m)q(n  +  m.n) 

=  Ml  ~  p)F({m}). 

where  F({m})  =  rm  \l-r),  m  >  1,  and  r  =  pA/(p+pX).  Thus,  if  X  is 
stationary.  then  Theorem  3.2(ii)  implies  that  H  is  Jty(A(l  -  p).F).  Hence 
D  is  a  compound  process  wi  lit  rale  A(  1  -  p)  and  geometric  atom 
distribution  F. 


4.  Poisson  Flows  in  a  Network  of  Queues 

We  shall  consider  a  queueing  network  process  defined  as  follows. 
Suppose  that  X( t )  =  (X j( t ) ,  .  . . ,X  (  l  )  ) ,  t  C  R,  is  a  queueing  network 
process  on  m  nodes,  where  X  ( l )  denotes  the  number  of  units  (i.e. 
customers)  at  node  ,j  at  time  t.  The  process  X  takes  on  values 

n  =  (n . n  )  in  '/.  .  Let  e.  denote  the  jth  unit  vector  in  7.m  with  1  in 

entry  j  and  0's  elsewhere.  As  in  Whittle  (1US(>),  we  assume  that  units 
move  among  the  m  nodes  such  that  X  is  a  Markov  process  with  transition 
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(4.1) 


q(n,n  +  ek)  =  XQk 

q(n,n  -  e  .  +  e,  )  =  X..<l>(n  -  e.)/4>(n).  n.  >  1 
J  k'  jk  j'  ’  J  “ 


q(n,n  -  e^)  =  A.0<*>(n  -  e  )/0(n). 


n  .  >  i , 
J  " 


and  q(n.n')  =  0  for  all  other  stales  n‘.  Here  <P-  Z™  -*  (0.<").  the  X's  are 

nonnegative,  and  the  subscript  0  denotes  the  "outside"  node.  Under  this 

assumption,  units  enter  the  nodes  1 . m  by  independent  Poisson 

processes  with  respective  rales  Xqj . ^Om'  ^en  X  is  in  state  n,  then 

2  X.,  <f>(n-e .  )/4>(n)  is  the  departure  rate  of  units  from  node  j.  The  X 

k  Jk  J  Jk 

is  the  "arc-dependent"  routing  intensity  from  node  j  to  node  k,  and 

<J>(n-e^.  )/<!>(  n)  is  the  "system-dependent"  departure  intensity  from  node  j 

(the  ratio  representing  the  potential  difference  between  the  system  in 

state  n  and  in  state  n-e^.  with  one  less  customer  at  node  j). 

We  shall  assume  that  X  is  irreducible.  This  is  equivalent  to  the 

irreducibi 1 i ty  of  the  Markov  routing  matrix 


(4.2) 


P  ( j  •  k )  =  *,,/  2  X  j.k  =  0 - m, 

JK  e=o 


where  X^  =  0.  The  irreducibi li ty  of  this  matrix  is  equivalent  to  the 
existence  of  unique  positive  numbers  w^ . tltat  satisfy 


(4.3) 


2  (WjAjk  -  *kXk.)  =0.  j  =  0 . m. 


m  n . 

where  w,.  =  1.  We  also  assume  that  2  <Hn)  17  w  is  finite.  Then  X  is 

0  .  ,  j 

n  J  =  1  J 

positive  recurrent  and  has  the  equilibrium  distribution  (p. 198  of  Whittle 


( 19SG) ) 


(4.4) 


m  n . 

tr(n)  =  c<J>(n)  TI  w  .  n  t  /.™. 

j  =  l  J 


where  c  is  the  normalizing  constant.  Hereafter,  we  assume  tltat  X  is 
st at  ionary . 
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Now,  consider  the  point  process 

N  (A)  =  2  1{X{ t)  =  X( L  — )  -  e  ) 

J  tt  A  J 

that  represents  the  times  at  which  units  exit  the  network  from  node  j. 

Of  course.  N  =0  when  X . „  =0. 

JO  jO 

Theorem  -4.1.  The  exit  processes  N,„ . N  are  independent  Poisson 

1U  mU 

processes  with  rates  w.X.„ . w  X  and  (N,„ . N  _)  X  . 

1  10  m  mO  v  10  mO'-  + 

Whittle  (1986)  on  p.  207  proved  this  by  establishing  that  the 

reverse  time  process  X  of  X  is  again  a  queueing  network  process  and  the 

exit  processes  of  X  are  just  the  time  reversals  of  the  Poisson  input 

processes  of  X  .  Theorem  4.1  also  follows  from  Theorem  3.2(ii)  and 

Remark  3.4  since,  for  each  n  and  j. 

^  _1 

a  (n. j } )  =  "(n)  *( n  +  e^)q(n  +  e_n)  =  WjXjo 

and  a  (n,(z))  =  0  elsewhere. 

Along  with  these  exit  processes,  consider  the  point  processes 

N  (A)  =  2  1 (X{ t )  =  X(t-)  -  e  +  e  ) 

J  tt  A  J  K 

of  limes  at  which  units  move  from  node  j  to  node  k.  We  shall  now 
iden  tify  sets  J  C  (1 . m)  and  K  C  (0 . 1  ,  .  .  .  ,m}  such  that  N  ...  jt  J .  ktK 

J  K 

are  independent  Poisson  processes.  Suppose  that  J  and  K  satisfy  the 
following  assumptions: 

A1 :  Each  unit  that  exits  J  can  never  return  to  J.  (To  verify  this  one 

need  only  check  the  possible  routing  under  X  ..  . ) 

Jk 

A2:  The  system-dependent  dejxirlure  intensity  for  each  node  jt J  is  of  the 


<t>(n  -  Cj)/^(n)  =  'f’jtrij  -  e.)/<t>j(nj) 


where  Oj  =  (ik  •'  jcj).  and  <J>j  is  a  positive  function  on  such  vectors. 


.s'--  \  .v  .s  n  \  •• (\  .v  • 
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A3:  K  is  the  largest  subset  of  {0,1 . m}  such  that  each  unit  in  K 

cannot  enter  J  on  a  subsequent  move.  (Note  that  0  €  K  and 

J  n  k  =  {j  €  j:  X  =  o,  e  €  j}. 

For  some  networks,  J  =  {1 . m}  may  be  the  only  set  of  nodes  that 

satisfies  A1 .  At  the  other  extreme  are  networks  in  which  each  node  is 

visited  at  most  once  by  a  unit,  and  so  each  subset  of  nodes  satisfies  A1  . 

Assumption  A2  is  equivalent  to  being  able  to  factor  <J>  as 

<t>(n)  =  <l>j(n j )<l'(n^  :  k€J).  Upon  selecting  J  conforming  to  A1 ,  A2,  it  is 

advantageous  to  select  K  as  large  as  possible  as  we  did  in  A3. 

Let  X.(t)  =  {X .( t ) : ji J} ,  ttR,  denote  the  process  X  on  the  nodes  J 
J  J 

and  let  denote  its  state  space. 

Theorem  4.2.  The  processes  N  ^  jtj,  ktK.  are  independent  Poisson 

processes  with  resi>ective  rates  w.A.,  .  jt.I,  ktK.  Furthermore, 

J  J  k 

{Njk:jtJ,  ktK}  _  Ji-  (Xj)  +  . 

Proof .  Under  the  assumptions,  the  Xj  is  a  queueing  network  process  on 
the  nodes  J  with  transition  rates  qj(n.n')  defined  as  in  (-1-1)  with  the 
last  line  replaced  by 

qj(n-n  -  e^)  =  A.<t>j(n  -  e^/^^n).  n.  >  1.  nt:)j, 

where  A.  =  2  A..  .  Furthermore,  X.  has  the  equilibrium  distribution 

J  ktK  Jk  J 

n  . 

ir.(n)  =  c  .<!>.(  n)  IT  a.',  nti  . . 

J  J  J  jtJ  J  J 

This  is  just  the  sum  of  w(n)  in  (4.4)  over  all  n^.  Pi J.  Thus,  from 

Theorem  4.1,  we  know  lliat  X.'s  exit  processes  N.  =  2  N  ,  i<.),  are 

J  J  ktK  -,k 

independent  Poisson  processes  with  respect ivo  rales  w^A^.  jtJ.  and  that 


(Nr  j*J)_  11  (X,)+. 


r.  V  VSWA'A-.VA  i 


Next,  observe  that  for  each  j  t  J,  the  processes  N  k  t  K.  form  a 

partition  of  N  .  in  which  each  point  of  N  .  is  assigned  to  the  subprocess 
J  J 

N .,  with  probability  A., /A..  independent  of  everything  else.  Conse- 
Jx  J*'  J 

quently,  N^.  k  e  K,  are  independent  Poisson  processes  with  respective 

rates  (A.  /A.)  (w.A.)  =  w.A...  kcK  (see  for  instance  p.89  of  Cinlar 

(197b)).  This  property  and  the  preceding  argument  yield  the  assertions. 0 

Example  4.3.  Suppose  the  network  lias  a  node  j  such  that  each  unit 

passing  through  the  network  visits  j  at  most  once.  Also,  assume  that 

4>(n)  =  <P  .(n .  )4>{n,  :  k  t  j).  and  let  J  =  (j).  Then  the  departure  processes 
J  J  * 

N  ...N.. . N.  are  independent  Poisson  processes  as  in  Theorem  4.2. 

JO  Jl  jm  ^ 

Now.  suppose  tliat  each  node  of  the  network  can  be  visited  at  most  once  by 

each  unit  and  <J>(n)  =  .  ( n .  ) .  .  .  <J>  (n  ).  Then  each  process  N  is  “/’(w.A..); 

some  of  these  processes  may  be  dependent .  Consider  the  arrival  processes 

to  a  fixed  node  k.  Lei  .1  denote  the  set  of  all  nodes  j  that  can  never  be 

reached  from  k.  Then,  under  the  preceding  supposition,  the  arrival 

processes  N  ,  jcj,  are  independent  Poisson  processes  as  in  Theorem  4.2. 

Example  4.4.  Multivariate  Poisson  Flows.  Suppose  .1^ . J  are  subsets 

of  (1 . m}  and  Kj . are  subsets  of  (0,1 . m)  (the  subsets  need 

not  be  disjoint).  Consider  the  process  N.  =  1  N .,  of  times  at  which 

1  jtjj  JK 


units  move  from  some  node  in  J.  to  some  node  in  K..  Suppose  the  sets 

1  i 

J  =  J.U...UJ  and  K  =  K.U.  .  .UK  satisfy  assumptions  Ai  ,  A2,  A3,  or  tlial 
1  v  1  II  ' 

they  are  contained  in  a  |xtir  of  such  sets.  Then  N  ,  i<  .1.  k<  K .  are 

jk  J 

independent  Poisson  processes  as  in  Theorem  4.2.  Consequently, 

(N, . N  )  is  a  p-dimensional  Poisson  process  with  rate  a  =  X  w.A.,  and 

1  ”  ■  ,  J  Jk 
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point  allocation  distribution 


*■■({  2  e  .})  =  a'1  yl  w  X.. 
itl  j.k  J  J 

where  I  C  {1 . l>).  and  the  sum  is  over  all  j  in  D  J .  ,  and  k  in  OK.. 

itl  1  itl  1 

The  Nj . N(  are  independent  if  ft  J.  =  $  and  fl  K.  =  4>. 

i  i  1 

Example  4.5.  Networks  With  Several  Types  of  Units.  Consider  a  network  as 

above  in  which  each  unit  carries  a  label  from  a  finite  set  ■J  of  types  and 

the  label  may  change  when  the  unit  moves.  We  represent  this  network  by 

the  process  X{t)  =  {X  ,(t):  atsf.  j  =  l . m}.  ttR.  where  X  (l)  is  the 

* 1 J  a  J 

number  of  type  a  units  at  node  j  at  time  t.  A  typical  state  of  X  is 


n  -  (n  . ati4.j=l . m) .  We  assume  tliat  X  is  a  Markov  process  with 

aj  K 

transition  rales  (analogous  to  (4.1)) 


c,(n.n  ♦  ebk)  =  XQ^ 

q(n.n  -  eaj  +  e^)  =  4>< n  -  e;j  .)/*{.,) , 


n  .  >  1  . 
a.)  - 


qfn.n  -  e  .)  =  X  ...  4»{n  -  e  .)/$(  n).  n  .  >  1. 

aj7  a  j ,  0  aj  v  '  a.) 

Under  assumptions  as  above,  the  process  X  lias  the  equilibrium 
distribution 

n  . 

ir(n  )  =  c4>(  n  )  n  w  n''  . 

aj  aJ 

The  results  above  hold  for  this  network  -  one  need  only  use  double 
indices  aj .  bk  in  place  of  j.k  and  consider  the  point  processes 
N^j  bk .  aj  t  J.  bk  t  K,  where  J,  K  satisfy  assumptions  Al,  A2.  AT  with 
double  indices.  For  instance,  suppose  A  C  rf  i s  such  that  a  unit  with  a 
label  from  jAA  can  never  carry  a  label  from  A.  Consider  the  point 
processes  of  times  at  which  units  chtut.ne  labels  from  A  to  .'AA.  These 


processes  are  bk .  ajC.I ,  bkCK ,  where  J  =  (aj:  at  A.  j=l . m)  and 

K  =  {bk:  bCjAA ,  k=0 . m)  .  Clearly  ,|  satisfies  Al  and  K  satisfies  AT. 


V 


Suppose  J  also  satisfies  A2.  Then  the  preceding  point  processes  are 


independent  Poisson  processes  with  respective  rates  (w  .A  .  } ,  and  the 

aj  a J  >  hk 

past  of  these  processes  is  independent  of  (Xj)+. 

The  preceding  results  are  for  networks  in  which  only  one  unit  can 
move  at  a  time.  In  networks  with  simultaneous  movement  of  units,  the 
flows  among  the  nodes  may  be  compound  Poisson  processes.  Kook  ( 1988)  has 
characterized  such  flows  us i tig  Theorem  3.2. 


5.  Further  Generalizations 

The  results  in  Sections  2  and  3  readily  extend  to  more  general 
processes.  We  discussed  some  of  these  situations  in  the  remarks  above. 
Here  are  some  more  generalizations. 

Harkov  Processes  With  General  State  Spaces.  Suppose  iliat  X  =  (X^  ttR)  is 

a  pure  jump  Markov  process  with  a  general  state  space  F.  and  associated 

o-field  £.  and  its  transition  kernel  is  K(x.B).  xeF.,  B(  £ .  Tliat  is,  the 

exponential  sojourn  time  in  state  x  has  parameter  K(x.K)  and  the 

probability  of  X  jumping  from  x  into  B  is  K(x,B)/K(x.E) .  Then  the 

results  in  Sections  2  and  3  hold  with  the  sums  replaced  by  integrals. 

For  example,  a  is  the  Kudon-N i kodym  derivative 

o*(x)  =  /  Tr(dy)K(y,dx)r(y.x)/7T(dx). 

i: 

Functionals  Involving  Sojourn  Times.  Suppose  X  is  a  Markov  process  as  in 
Sect  ion  2  and 

N(A)  =  X  f(X  X  W  ) 
tcA 

where  is  the  wailing  lime  in  state  X^_  and  f : ,'t  x  :f  x  -*  {0,1}.  Then 

if 

the  results  of  Section  2  hold  for  this  N  with  a.  o  defined  by 


V  ,v  V,v>  v.v.v.va  , v.N 


2. 


.  UJ 

q(x)  =  2  /0  q(x.y)f(x.y.w)Fx(dw) 

y 

a*(x)  =  tt ( x )  */“  >:  Tr(y)q(y.x)f(y.x.v»)F  (dw) 

y 

where  F^fw)  =  1  **  exp(-wq(x)).  Similarly,  the  results  of  Section  3  hold 
for  M  as  a  functional  of  as  well  as  of  Xf .X^  .  One  can  generalize 
further  by  assuming  that  X  is  a  semi-Markov  process  and  replacing  F^  by  a 
general  sojourn  time  distribution  F 
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